Method for measuring objects in digestive tract based on imaging system

ABSTRACT

A method for measuring objects in a digestive tract based on an imaging system is provided. The imaging system captures a detection image in the measurement stage. The depth distance z i  from a target point P′ to a board is calculated, a correction factor is obtained according to the predicted brightness g −1 (z i ) of a reference point P in the detection image. The depth image z(x, y) from the actual position of each pixel to the board is calibrated by the correction factor. The scale r of each pixel is calculated according to the depth image z(x, y). The actual two-dimensional coordinates S i ′ of each pixel are calculated by the scale r, and the actual three-dimensional coordinates (S i ′, z(x, y)) of each pixel are obtained. The distance between any two pixels in the detection image or the area within any range are calculated.

CROSS-REFERENCE OF RELATED APPLICATIONS

The application claims priority to Chinese Patent Application No.201910347967.X filed on Apr. 28, 2019, the contents of which areincorporated by reference herein.

FIELD OF INVENTION

The present invention relates to image processing, and more particularlyto a method for measuring objects in digestive tract based on an imagingsystem.

BACKGROUND

At present, capsule endoscopes have become increasingly popular fordigestive tract examination. Doctors examine subjects through imagestaken by the capsule endoscopes inside the digestive tract of thesubjects. However, for a normal capsule endoscope, when a captured imageshows a lesion, the doctor can only determine through the shape, color,position and other characteristics of the lesion, but cannot obtain thesize information of the lesion, and therefore cannot give an accuratequantitative analysis result.

In the prior art, for example, according to a Chinese Patent No.CN107072498A, a method for distance measuring in the digestive tract isprovided. In the above method, some distance measuring pixels aredistributed in a common image sensor to generate distance measuring datafor object depth information, that is depth image. Then, the distancemeasuring data of known pixel positions is used to interpolate the depthof the pixel position of the distance measuring data not derived in thedepth image. In the above method, the calculation method is complicated,and the requirements on components are high. The distance measuringpixels need to be evenly distributed in the image sensor, and moremeasurement points are required. If there are a plurality of measurementpoints, a large deviation can appear in subsequent gradientcalculations, which can eventually cause measurement distortion.

According to a Chinese Patent No. CN101902961A, a device, system andmethod for estimating the size of an object in a body lumen is provided.In the above method, a laser device is configured in the capsuleendoscope, and distance measurement and object measurement are performedby laser points and image brightness. However, the effect of media inthe digestive tract is ignored in the method. The environment of thedigestive tract is complex. Both air and digestive fluid can affect theoptical path of laser and directly affect the result of laser distancemeasuring. In addition, the distance measurement can always depend onthe results of laser distance measuring. Each measurement requires laserdistance measuring, and a plurality of calculations are required at eachtime of measurement, which consumes manpower and material resources.

Furthermore, in the prior art as described above, the size is measuredwith the aid of additional device, such as a time of flight (ToF)distance-measuring chip, which undoubtedly increases the cost andcomplexity of measuring system.

Therefore, it is necessary to design a new method for measuring objectsin the digestive tract with more convenient calculation steps.

SUMMARY OF THE INVENTION

To solve one of the above problems, the present invention provides amethod for measuring objects in digestive tract based on an imagingsystem, comprising: simulating an environment in the digestive tract andentering a calibration stage of the imaging system;

setting a plurality of calibration points Q′ on a transparent enclosureof the imaging system;

controlling a photographing unit of the imaging system to photograph andform a calibration image, and recording the calibration point Q′ imagedin the calibration image as an imaging point Q;

calculating and determining the relationship between the relative angleθ of the calibration point Q relative to the optical axis of thephotographing unit and the pixel distance Δq′ from the imaging point Qto the center of the calibration image, and recording it as:θ=f(Δq′)  (1);

calculating and determining the relationship between the brightness φ ofany pixel in the calibration image and the depth distance z from theactual position of the pixel in the simulated digestive tract to a boardwhere the photographing unit is disposed, and recording it as:z(x,y)=g(φ(x,y))  (2);

-   -   calculating the relationship between the scale r of any pixel in        the calibration image and the depth distance z from the actual        position of the pixel in the simulated digestive tract to the        board where the photographing unit is disposed, where scale is        the actual length represented by unit pixel in the image, and        recording it as:        r=dz  (3);

entering a measurement stage after calibration is completed;

placing the imaging system in the digestive tract;

capturing and obtaining a detection image;

determining a region in the detection image where the transparentenclosure contacts with the digestive tract wall, and recording it as acontact region, and setting at least one reference point P in thecontact region, and recording the actual position of the reference pointin the digestive tract as a target point P′;

calculating the pixel distance Δp from the reference point P to thecenter of the detection image separately, and putting it into equation 1to obtain the relative angle θ of the target point P′ relative to theoptical axis of the photographing unit;

calculating the actual distance from the target point P′ to the boardseparately and recording it as depth distance z_(i);

obtaining the predicted brightness g⁻¹ (z_(i)) of the reference point Pin the detection image according to equation 2 and the depth distancez_(i);

comparing the predicted brightness g⁻¹(z_(i)) of the reference point Pwith the actual pixel brightness img(P_(i)) of the reference point P toobtain a correction factor k_(i), and recording it as:

$\begin{matrix}{{k_{i} = \frac{g^{- 1}\left( z_{i} \right)}{{img}\left( P_{i} \right)}};} & (4)\end{matrix}$

obtaining a mean value k of the correction factors k_(i) of allreference points P;

calibrating all pixels in the detection image with the mean value k ofthe correction factors to obtain the depth distance from the actualposition of each pixel in the digestive tract to the board, andrecording it as depth image z(x, y), where:z(x,y)=g( k img(x,y))  (5);

calculating the scale r of each pixel in the detection image accordingto equation 3 and the depth image z(x, y);

obtaining the pixel coordinates S_(i) of each pixel point in thedetection image, and calculating the actual two-dimensional coordinatesS_(i)′ of each pixel in the detection image by the scale r;

integrating to obtain the actual three-dimensional coordinates (S_(i)′,z(x, y)) of each pixel;

calculating or measuring the distance between any two pixels in thedetection image or the area within any range.

In an embodiment, the step “determining the region in the detectionimage where the transparent enclosure contacts with the digestive tractwall” comprises: selecting the edge part of the detection image awayfrom the center of the detection image;

obtaining the brightness T of each pixel in the edge part;

gathering the pixels in a region which the brightness T is greater thana threshold τ as the contact region.

In an embodiment, the step “selecting the edge part of the detectionimage away from the center of the detection image” comprises:

marking an inner ring on the detection image that is centered on thecenter of the detection image, wherein the inner ring is close to theedge of the detection image and does not intersect;

marking an outer ring on the detection image that is centered on thecenter of the detection image, wherein the outer ring intersects withthe edge of the detection image;

recording the part enclosed by the inner ring, the outer ring, and theimage edge as the edge part.

In an embodiment, the digestive tract comprises a plurality of regionsand the imaging system comprises a plurality of exposure levels, andwherein, after the step “obtaining the mean value k of the correctionfactors k_(i) of all reference points P”, the correction factors arestored according to different exposure levels and different digestivetract regions.

In an embodiment, after two or more mean values k of correction factorsare obtained at the same exposure level and digestive tract region, theaverage of the mean values k of correction factors is calculated beforestoring and updating.

In an embodiment, the step “calculating the actual distance from thetarget point P′ to the board separately and recording it as depthdistance z_(i) of the reference point P” comprises:

obtaining the radius R of a front enclosure of the transparentenclosure; calculating the distance R cos θ from the target point P′ tothe spherical center of the front enclosure separately;

obtaining the axial length H of an annular enclosure of the transparentenclosure; calculating the depth distance z_(i)=R cos θ+H from thetarget point P′ to the board separately.

In an embodiment, the step “obtaining the depth distance from the actualposition of each pixel in the digestive tract to the board” or“integrating to obtain the actual three-dimensional coordinates(S_(i)′,z(x,y)) of each pixel” further comprises:

determining the value of the depth distance z of each pixel;

when t₁≤z≤t₂, it is determined that the pixel is within the effectivesection of the detection image;

when z<t₁ or z>t₂, it is determined that the pixel is within theineffective section of the detection image.

In an embodiment, the step “calculating or measuring the distancebetween any two pixels in the detection image or the area within anyrange” is followed by:

calculating a straight-line distance between any two pixels selected bya user in the effective section according to the three-dimensionalcoordinates of the two pixels; or,

building a three-dimensional image of any area according to thethree-dimensional coordinates of pixels in the area selected by a userin the effective section, and calculating a straight-line distancebetween any two pixels selected by the user from the three-dimensionalimage; or,

calculating the area of any region selected by a user in the effectivesection according to the three-dimensional coordinates of the region;or,

forming a scale in the effective section, and marking graduations on thescale as those of actual length; or,

identifying the lesion region in the effective section automatically,and calculating the size or area of the region.

In an embodiment, t₁=0, and t₂=60 mm.

In an embodiment, the step “capturing and obtaining a detection image”comprises:

controlling the imaging system to capture and obtain an image;

correcting the radial distortion of the captured image and forming adetection image, and recording it as:img_out(x,y)=img_in(x(1+l ₁ R ² +l ₂ R ⁴),y(1+l ₁ R ² +l ₂ R ⁴))  (6);

where, R=√{square root over (x²+y²)} represents the pixel distance fromthe pixel to the center of the detection image, l₁ and l₂ representdistortion parameters of the imaging system, x represents x-coordinateof the pixel, y represents y-coordinate of the pixel, img_in representsinput image, and img_out represent corrected image.

The present invention further provides a measuring system for objects indigestive tract based on an imaging system, comprising:

an identification module, configured to identify a contact region and areference point P;

a calibration calculation module, configured to calculate therelationship between the relative angle θ of a calibration point Q′relative to the optical axis of a photographing unit of the imagingsystem and the pixel distance Δq′ from an imaging point Q to the centerof a calibration image and record as equation 1, and to calculate therelationship between the brightness φ of any pixel in the calibrationimage and the depth distance z from the actual position of the pixel ina simulated digestive tract to the imaging system, and record it asequation 2, and to calculate and determine the relationship between thescale r of any pixel in the calibration image and the depth distance zfrom the actual position of the pixel in the simulated digestive tractto the imaging system, and record it as equation 3;

a brightness detection module, configured to identify the brightness ofany pixel or all pixels in the calibration image or a detection image;

a measurement calculation module, configured to obtain the equation 1and the pixel distance Δp from the reference point P to the center ofthe detection image to calculate the relative angle θ of a target pointP′ relative to the optical axis of the photographing unit, and tocalculate actual distance from the target point P′ to a board where thephotographing unit is disposed and record it as depth distance z_(i),and to obtain the depth distance z_(i), the equation 2 and the actualpixel brightness of the reference point P to calculate a correctionfactor k_(i), and to obtain the equation 2 to calculate and obtain thedepth distance z(x, y) from the actual position of each pixel in thedigestive tract to the board, and to obtain the equation 3 to calculatethe actual two-dimensional coordinates S_(i), and integrate to obtainthe actual three-dimensional coordinates of (S_(i)′, z (x, y)) of eachpixel.

Compared to the prior art, the method for measuring objects in thedigestive tract based on the imaging system in the present invention canobtain some parameter information in advance through the calibrationstage of the imaging system, and thereby facilitate the calculation inthe measurement stage and avoid calculation error caused by equipmentdifference between imaging systems. Moreover, by determining the contactregion, the reference point in the captured image can directlycorrespond to the target point on the transparent enclosure of theimaging system, so that no other hardware is needed to measure the depthdistance z_(i) for reference point, making the components simpler andthe calculation steps more concise.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an exemplary flowchart of the measurement method in acalibration stage of the present invention.

FIG. 2 shows an exemplary flowchart of the measurement method in ameasurement stage of the present invention.

FIG. 3 shows a structural diagram of a partial structure of an imagingsystem according to the present invention.

FIG. 4 shows a schematic diagram of an image captured by the imagingsystem according to the present invention.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand thetechnical solutions disclosed, the present invention can be described indetail below with reference to the accompanying drawings and preferredembodiments. However, the embodiments are not intended to limit theinvention, and obviously, the described embodiments are only a part ofthe embodiments of the present invention, but not all of them. All otherembodiments obtained by those having ordinary skill in the art withoutcreative work based on the embodiments of the present invention areincluded in the scope of the present invention.

Referring to FIGS. 1-4 , the present invention provides a method formeasuring objects in digestive tract based on an imaging system and oneor more computer processors, where the imaging system can be introducedinto the digestive tract to take images of the objects therein, and atwo-axis coordinate system is preset on the images taken by the imagingsystem to determine the coordinates of any pixel in the image. In theembodiment, the imaging system is a capsule endoscope. The capsuleendoscope comprises a board 104, a photographing unit 102 disposed onthe board 104, and a transparent enclosure disposed around thephotographing unit 102. The photographing unit 102 can take imagesthrough the transparent enclosure. If the imaging system is otherequipment, as long as the components are included, the object of thepresent invention can also be achieved.

Moreover, since the imaging system, especially the capsule endoscope inthe embodiment, is in the digestive tract, the transparent enclosuregenerally comes into contact with the inner wall of the digestive tract.Especially in esophagus and large intestine, the lumen space is smallbecause of insufficient water. During the peristalsis of colon andswallowing of the esophagus, the inner wall of colon and esophagus canwrap and squeeze the capsule endoscope, and the inner wall can usuallycome into contact with the transparent enclosure. In the smallintestine, due to its curved structure, smaller inner diameter, and morefrequent contraction, the inner wall can also contact with thetransparent enclosure. Therefore, in the esophagus, small intestine, andlarge intestine, it can be assumed that the inner wall of the digestivetract is in contact with the imaging system, so that the captured imagehas a part where the transparent enclosure contacts with the digestivetract.

Specifically, the measurement method comprises:

simulating an environment of the digestive tract and entering acalibration stage of the imaging system;

setting a plurality of calibration points Q on the transparent enclosureof the imaging system;

controlling the photographing unit 102 of the imaging system tophotograph and form a calibration image, and recording the calibrationpoint Q′ imaged in the calibration image as an imaging point Q;

calculating and determining the relationship between the relative angleθ of the calibration point Q′ relative to the optical axis of thephotographing unit 102 and the pixel distance Δq′ from the imaging pointQ to the center of the calibration image, and recording it as:θ=f(Δq′)  (1);

calculating and determining the relationship between the brightness φ ofany pixel in the calibration image and the depth distance z from theactual position of the pixel in the simulated digestive tract to theboard 104 where the photographing unit 102 is disposed, and recording itas:z(x,y)=g(φ(x,y))  (2);

calculating the relationship between the scale r of any pixel in thecalibration image and the depth distance z from the actual position ofthe pixel in the simulated digestive tract to the board 104 where thephotographing unit 102 is disposed, where scale is the actual lengthrepresented by unit pixel in the calibration image, and recording it as:r=dz  (3);

entering a measurement stage after calibration is completed;

placing the imaging system in the digestive tract;

capturing and obtaining a detection image;

determining the region in the detection image where the transparentenclosure contacts with the digestive tract wall, and recording it as acontact region, and setting at least one reference point P in thecontact region, and recording the actual position of the reference pointin the digestive tract as a target point P′;

calculating the pixel distance Δp from the reference point P to thecenter of the detection image separately, and putting it into equation 1to obtain the relative angle θ of the target point P′ relative to theoptical axis of the photographing unit 102;

calculating the actual distance from the target point P′ to the board104 separately and recording it as a depth distance z_(i);

obtaining the predicted brightness g⁻¹(z_(i)) of the reference point Pin the detection image according to equation 2 and the depth distancez_(i);

comparing the predicted brightness g⁻¹(z_(i)) of the reference point Pwith the actual pixel brightness img(P_(i)) of the reference point P toobtain a correction factor k_(i), and recording it as:

$\begin{matrix}{{k_{i} = \frac{g^{- 1}\left( z_{i} \right)}{{img}\left( P_{i} \right)}};} & (4)\end{matrix}$

obtaining a mean value k of the correction factors k_(i) of allreference points P;

calibrating all pixels in the detection image with the mean value k toobtain the depth distance from each pixel to the board 104, andrecording it as the depth image z(x, y), where:z(x,y)=g( k img(x,y))  (5);

calculating the scale r of each pixel in the detection image accordingto equation 3 and the depth image z(x, y);

obtaining the pixel coordinates S_(i) of each pixel point in thedetection image, and calculating the actual two-dimensional coordinatesS_(i)′ of each pixel in the detection image by the scale r;

integrating to obtain the actual three-dimensional coordinates (S_(i)′,z(x, y)) of each pixel;

calculating or measuring the distance between any two pixels in thedetection image or the area within any range.

In the above method, calibration is first performed in the calibrationstage. The first step is to obtain the relationship between the relativeangle θ of the calibration point Q′ relative to the optical axis of thephotographing unit 102 and the pixel distance Δq′ from the imaging pointQ to the center of the calibration image. The second step is to obtainthe relationship between the brightness φ of any pixel in thecalibration image and the depth distance z from the actual position ofthe pixel in the simulated digestive tract to the board 104 where thephotographing unit 102 is disposed. The third step is to obtain therelationship between the scale r of any pixel in the calibration imageand the depth distance z from the actual position of the pixel in thesimulated digestive tract to the board. Then, in the actual measurementstage, determining a region in the detection image where the digestivetract is in contact with the transparent enclosure. The target point P′in the contact region is both a point on the transparent enclosure and apoint within the digestive tract. Therefore, the relative angle θ of thetarget point P′ relative to the optical axis of the photographing unit102 can be obtained through the reference point P of the target point P′in the detection image. Then the actual distance from the target pointP′ to the board 104 can be calculated through the structure of theimaging system, where the actual distance is the depth distance z_(i).Then the predicted brightness g⁻¹(z_(i)) of the reference point P can becalculated. Then, the predicted brightness g⁻¹(z_(i)) of the referencepoint P is compared with the actual pixel brightness img(P_(i)) of thereference point P to obtain the correction factor k_(i). After obtainingthe correction factor k_(i), all pixels in the detection image arecorrected to obtain the predicted brightness of each pixel, so as toobtain the depth distance z(x, y) from each pixel to the imaging system.Finally, the actual two-dimensional coordinates S_(i)′ on the xoy planeof each pixel are obtained through the scale. The information describedabove is integrated to obtain the actual three-dimensional coordinates(S_(i)′, z(x, y)) of each pixel. Therefore, after knowing the actualthree-dimensional coordinates of each pixel, the distance between anytwo pixels in the detection image or the area within any range can becalculated.

Moreover, assuming that the transparent enclosure is in contact with thedigestive tract, the target point P′ in the digestive tract is thetarget point P′ on the transparent enclosure, so that the actualcoordinates of pixels in the captured image can be directly obtainedaccording to the internal structure of the imaging system, with no needfor the structure of other components, and the overall structure of theimaging system is relatively simple.

The actual pixel brightness img(P_(i)) of the reference point P is thebrightness of the pixel at point P in the detection image. Since theform of function g can be related to the reflection coefficient of theobject surface, exposure parameters, media environment, number of LEDs,distribution, camera lens performance of the photographing unit 102 andthe response of image sensor of the photographing unit 102, although therelationship between the brightness φ of any pixel in the calibrationimage and the depth distance z from the actual position of the pixel inthe simulated digestive tract to the board where the photographing unitis disposed is obtained during calibration, when the actual distancez_(i) and the predicted brightness g⁻¹(z_(i)) are obtained in subsequentprocess, it still needs to be compared with the actual pixel brightnessof the reference point P to obtain a correction factor k_(i) to correctthe actual brightness of other pixels to obtain the depth distance z ofsaid other pixels.

In addition, in the final calculation process, two pixels or any areacan be selected manually or by a system, and then measured by thesystem; or, the system provides a scale, and the values are directlyread or measured manually.

Therefore, each imaging system needs to first enter the calibrationstage to measure and determine different parameters of the imagingsystem. Accordingly, even though there is difference between imagingsystems, different parameters of the imaging system can be obtained inthe process of calibration, and the parameters are needed formeasurement and calculation in the subsequent process, so as to avoiderrors due to difference in equipment.

It should be noted that, in the embodiment, when the photographing unit102 takes an image, the optical axis of the photographing unit 102 isthrough the center of the image, and the connecting line between theoptical axis and the center of the image is recorded as a referenceline, that is the direction of z axis. Therefore, the depth distancefrom the reference point P in the image to the imaging system does notrefer to a linear distance between the two, but refers to the distancebetween the two in line with the direction of the reference line. Inaddition, the image taken by the imaging system is preset with atwo-axis coordinate system recorded as a xoy plane coordinate system.The pixel coordinates S_(i) and the actual two-dimensional coordinatesS_(i)′ of each pixel are based on the xoy plane. Then the depth image z(x, y) is obtained, and the two is combined into the three-dimensionalcoordinates.

First of all, during calibration, the imaging system needs to be placedin a calibration box. The calibration box is a dark chamber and ensuresopacity to light. The calibration box comprises a fixing frame forfixing the imaging system, a target board for the imaging system to takeimages thereon, and the calibration box is also filled with a simulationmedium that can be simulated digestive liquid or air. The imaging systemcan move on the fixing frame. The imaging system further comprises aplurality of LEDs, where the LEDs and the photographing unit 102 areboth arranged on the inner board of the capsule endoscope, and thenumber of the LEDs is set to be 2 to 5, and they are distributed aroundthe photographing unit 102. Therefore, the imaging system can be set totake images at different positions, under different lighting conditions,in different simulation media and on different target boards to obtainthe parameter information. The target board can also be replaced, suchas a hard board simulating the mucosal surface or imitating the color ofmucosa. When the calibration box is used for other calibrations, onlythe target board needs to be replaced with a whiteboard, a chess boardor a line pairs card, so that white balance correction, camera parametercalibration, resolution measurement and other calibrations can beperformed. The light field distribution of the LEDs can affect thedistribution of the brightness φ of any pixel in the calibration image,so each different imaging system must be calibrated separately.

Moreover, during calibration, after each image is obtained, a radialdistortion correction for the image is required. This is because thecapturing of image can be affected by the distortion parameters ofdifferent cameras. Therefore, distortion correction can improve theaccuracy of size calculation of objects on the image, especially themeasurement of objects at the edge of the image. The image with radialdistortion correction can be calibrated to obtain parameter information.The specific information of radial distortion correction can bedescribed in detail later.

In the measurement stage, as the correction factor is obtained, allpixels in the image can be calibrated and the actual depth distance z(x, y) from the actual position of each pixel in the digestive tract tothe board 104 can be obtained. Due to different photographingenvironments of the imaging system and different positions in thedigestive tract, the correction factor can be affected accordingly.Specifically, the digestive tract comprises a plurality of regions andthe imaging system comprises a plurality of exposure levels according todifferent photographing environments. So, after the step “obtaining themean value k of the correction factors k_(i) of all reference points P”,the mean value k of the correction factors should be stored according todifferent exposure levels and different digestive tract regions, andupdated to a correction factor k. In the same digestive tract region andwith the same exposure parameters, the correction factors k have smallgap, so even if there is no reference point P, the depth distance z(x,y) from each pixel to the imaging system can also be obtained through apredicted correction factor. This method can not only improve theanti-interference ability of the entire system algorithm, but alsoreduce the number of images taken with the reference point P, thusimproving work efficiency.

If two or more correction factors k are obtained at the same exposurelevel and in the same digestive tract region, the average of thecorrection factors k should be calculated before storing and updating.Specifically, as shown in table 1, the digestive tract regions includeesophagus, small intestine, large intestine, etc., the exposure levelsinclude 1, 2, . . . to N, and different exposure levels and digestivetract regions are stored with different correction factors k. Therefore,if the reference point P is not obtained to calculate the correctionfactor k, the corresponding correction factor can also be selected forcalculation from the table below according to the exposure level anddigestive tract region.

TABLE 1 Digestive tract regions Exposure Small Large levels Esophagusintestine intestine 1 k11 k12 k13 2 k21 k22 k23 . . . . . . . . . . . .N kN1 kN2 kN3

As described above, the premise of the present invention is to assumethat the inner wall of the digestive tract is in contact with thetransparent enclosure and analyze the target point P′ in the contactregion. Therefore, how to determine the contact region in the detectionimage is a difficult point. The step “determining the region in thedetection image where the transparent enclosure contacts with thedigestive tract wall” comprises:

selecting the edge part of the detection image away from the center ofthe detection image;

obtaining the brightness T of each pixel in the edge part;

gathering the pixels in a region which the brightness T is greater thana threshold τ as the contact region.

Obviously, due to the concentrated propagation of light beams emittedfrom the LEDs of the imaging system and the light reflection on theinner wall of the digestive tract, there is a clear brightness stepdifference in the images taken by the imaging system. When the target iscloser to the transparent enclosure, the brightness of the target in thecorresponding image is higher, and when the target is farther from thetransparent enclosure, the brightness of the target in the correspondingimage is lower.

Therefore, the photographing environments in the digestive tract can besimulated in the early simulation experiment stage to calculate thebrightness distribution of the contact region, so as to derive thethreshold τ. The region composed of all pixels greater than thethreshold τ can be regarded as the contact region.

In addition, determination of the contact region by the threshold τ mayhave an error. For example, omitting some contact points results in adetermination with omission, or treating some misjudged points that arenot in contact but are closer as contact points results in amisjudgment. But the determination with omission can basically not causea calculation error. If a misjudgment occurs, the misjudged points thatare not actually contacted are determined as contact points, which maycause an error. However, the range of the above error cannot be verylarge and can be ignored, mainly due to the following three reasons.First, the threshold τ can be a relatively large value, and under thethreshold τ, the above determination is more likely to be adetermination with omission but not misjudgment. Second, even if themisjudged points do not contact with the transparent enclosure, they areusually very close to the transparent enclosure, so the above error canbe ignored. Third, in the above steps, at least one reference point Pcan be selected in the contact region, and in the actual operationprocess, in order to make the mean value k of correction factorsaccurate, a large number of reference points P can be taken, so that theabove error can be further reduced after averaging.

Further, in the above steps, the edge part of the detection image awayfrom the center of the detection image should be selected. Due todigestive tract peristalsis, the transparent enclosure of the imagingsystem usually contacts with the inner wall of the digestive tract atits edge part. Therefore, the contact region is usually formed at theedge port of the detection image. Specifically, the step “selecting theedge part of the detection image away from the center of the detectionimage” comprises:

marking an inner ring on the detection image that is centered on thecenter of the detection image, and the inner ring is close to the edgeof the detection image and does not intersect;

marking an outer ring on the detection image that is centered on thecenter of the detection image, and the outer ring intersects with theedge of the detection image;

recording the part enclosed by the inner ring, the outer ring, and theimage edge as the edge part.

Therefore, the range of the edge part can be determined, and the rangecannot be too large, causing difficulty in determination of thethreshold τ, nor too small, causing difficulty in selecting thereference point P. The radius of the inner ring and the outer ring isdetermined by the size of the detection image.

As described above, during calibration, to ensure image accuracy, aftereach image is obtained, a radial distortion correction for the image isrequired. Therefore, in the specific implementation process of thepresent invention, a radial distortion correction is also required forthe captured images. Specifically, the step “capturing and obtaining adetection image” comprises:

controlling the imaging system to capture and obtain an image;

correcting the radial distortion of the captured image and forming adetection image, and recording it as:img_out(x,y)=img_in(x(1+l ₁ R ² +l ₂ R ⁴),y(1+l ₁ R ² +l ₂ R ⁴))  (7);

where, R=√{square root over (x²+y²)} represents the pixel distance fromthe pixel to the center of the detection image, l₁ and l₂ representdistortion parameters of the imaging system, x represents x-coordinateof the pixel, y represents y-coordinate of the pixel, img_in representsinput image, and img_out represent corrected image.

Therefore, after correcting the radial distortion of the captured image,the detection image is obtained, and then an edge part is selected onthe detection image to obtain the contact region and the reference pointP.

Further, as shown in FIG. 3 , after a reference point P is obtained, therelative angle θ between the reference point and the optical axis of thephotographing unit 102 can be obtained in the calibration stage.Specifically, as shown in FIG. 3 , the transparent enclosure comprises afront enclosure 103 and an annual enclosure 106 connecting the frontenclosure 103 to the board 104, where a vertex of the front enclosure103 is located on the optical axis of the photographing unit 102. Then,point A is designated as the vertex of the front enclosure 103, and theimaging point A′ of the point A in the captured image is the center ofthe image. The target point P′ is also on the front enclosure 103, andthe target point P′ forms a reference point P in the detection image105. Point O is the spherical center of the front enclosure 103. It isknown that the radius R of the front enclosure 103 and the axial lengthH of the annular enclosure 106, the relative angle θ between thereference point and the optical axis of the photographing unit 102 isthe acute angle between the line connecting the reference point and thespherical center of the front enclosure 103 and the optical axis of thephotographing unit 102. That is, θ=∠AOP′. The step “calculating theactual distance from the target point P′ to the board 104 separately andrecording it as depth distance z_(i) of the reference point P”comprises:

obtaining the radius R of the front enclosure 103 of the transparentenclosure; calculating the depth distance R cos θ from the target pointP′ to the spherical center of the front enclosure 103 separately;

obtaining the axial length H of the annular enclosure of the transparentenclosure; calculating the depth distance z_(i)=R cos θ+H from thetarget point P′ to the board 104 separately.

The coordinates of the target point P′ in the z-axis direction is R cosθ+H.

Further, to facilitate subsequent calculation in detail, the actualcoordinates of the target point P′ in the xoy plane can also becalculated. Specifically, the distance between the target point P′ andthe point A in the xoy plane is R sin θ. Referring to FIG. 4 , sincepoint A′ is the center of the image, point A is set as the origin of thecoordinate system of xoy plane, and it is assumed that the angle betweenthe target point P′ and the origin A is α. Then, as described above, theactual coordinates of the target point P′ in the xoy plane are: x=R sinθ cos α; y=R sin θ sin α. Then, the actual three-dimensional coordinatesof the target point P′ are (R sin θ cos α, R sin θ sin α, R cos θ+H).After the actual three-dimensional coordinates of the target point P′are obtained, the correction factors and the depth distance from theactual position of other pixels in the digestive tract to the board 104can be obtained.

However, even if the depth distance from the actual position of eachpixel in the digestive tract to the board 104 is obtained, if the objectis too far from the capsule endoscope, the captured image can be toodark, and a large error can be easily caused in the depth distance zcalculated according to the image brightness in equation 2; moreover,the image may be blurred with reduced resolution and larger noise, andthe calculation error of the depth distance z becomes greater.Therefore, an effective section of the detection image must be defined,and only the detection image within the effective section can be usedfor measurement and calculation.

Therefore, after obtaining the depth distance or the depth image, it isnecessary to compare the depth distance z. Specifically, the step“obtaining the depth distance from the actual position of each pixel indigestive tract to the board 104” or “integrating to obtain the actualthree-dimensional coordinates (S_(i)′, z(x,y)) of each pixel” furthercomprises:

determining the value of the depth distance z of each pixel;

when t₁≤z≤t₂, it is determined that the pixel is within the effectivesection of the detection image;

when z<t₁ or z>t₂, it is determined that the pixel is within theineffective section of the detection image.

where, t₁=0, and t₂=60 mm.

Therefore, after the final step “calculating or measuring the distancebetween any two pixels in the detection image or the area within anyrange”, any two pixels or any range taken are also within the effectivesection of the detection image.

Specifically, in a first interaction mode, a straight-line distancebetween any two pixels selected by a user in the effective section canbe calculated according to the three-dimensional coordinates of the twopixels.

Or, in a second interaction mode, a three-dimensional image of anyregion can be built according to the three-dimensional coordinates ofpixels in the region selected by a user in the effective section, and astraight-line distance between any two pixels selected by the user fromthe three-dimensional image can be calculated.

Or, in a third interaction mode, the area of any region selected by auser in the effective section can be calculated according to thethree-dimensional coordinates of the region.

Or, in a fourth interaction mode, a scale is formed in the effectivesection, and the graduations on the scale are marked as those of actuallength, users can place the scale at different positions, and thegraduations of the scale at different positions are also different, thenusers can read and measure by themselves.

Or, in a fifth interaction mode, the lesion region in the effectivesection can be automatically identified, with the size or area of theregion calculated.

The above step “calculating the distance between any two pixels in thedetection image or the area within any range” is not limited to haveonly the five interaction modes, but the calculation method is based onthat the actual three-dimensional coordinates of each pixel have beenobtained, so other interaction modes are also within the protectionscope of the present invention.

Therefore, accordingly, the present invention further provides ameasuring system for objects in the digestive tract based on an imagingsystem, comprising: an identification module, configured to identify thecontact region and the reference point P;

a calibration calculation module, configured to calculate therelationship between the relative angle θ of the calibration point Q′relative to the optical axis of the photographing unit and the pixeldistance Δq′ from the imaging point Q to the center of the calibrationimage and record as equation 1, and to calculate the relationshipbetween the brightness φ of any pixel in the calibration image and thedepth distance z from the actual position of the pixel in the simulateddigestive tract to the imaging system, and record it as equation 2, andto calculate and determine the relationship between the scale r of anypixel in the calibration image and the depth distance z from the actualposition of the pixel in the simulated digestive tract to the imagingsystem, and record it as equation 3; a brightness detection module,configured to identify the brightness of any pixel or all pixels in thecalibration image or the detection image;

a measurement calculation module, configured to obtain the equation 1 ofthe calibration calculation module and the pixel distance Δp from thereference point P to the center of the detection image to calculate therelative angle θ of the target point P′ relative to the optical axis ofthe photographing unit 102, and to calculate actual distance from thetarget point P′ to the board and record it as depth distance z_(i), andto obtain the depth distance z_(i), the equation 2 of the calibrationcalculation module and the actual pixel brightness of the referencepoint P to calculate the correction factor k_(i), and to obtain theequation 2 to calculate and obtain the depth distance z(x, y) from theactual position of each pixel in the digestive tract to the board, andto obtain the equation 3 to calculate the actual two-dimensionalcoordinates S_(i)′, and integrate to obtain the actual three-dimensionalcoordinates of (S_(i)′, z(x, y)) of each pixel.

In summary, the method for measuring objects in the digestive tractbased on the imaging system in the present invention can obtain someparameter information in advance through the calibration stage of theimaging system, and thereby facilitate the calculation in themeasurement stage and avoid calculation error caused by equipmentdifference between imaging systems. Secondly, by the storage ofcorrection factors k_(i), after the storage amount of k_(i) is gettinglarger and the value of k_(i) is getting more stable, the correctionfactor k_(i) may not be calculated in the subsequent photographingprocess, so the use of a distance measuring unit can be ignored.Moreover, by determining the contact region, the reference point in thecaptured image can directly correspond to the target point on thetransparent enclosure of the imaging system, so that no other hardwareis needed to measure the depth distance z_(i) for reference point,making the components simpler and the calculation steps more concise.

Finally, through separate measurement in different digestive tractenvironments in the calibration stage, different processing methods canbe selected for different digestive tract environments to improveaccuracy.

It should be understood that, although the specification is described interms of embodiments, not every embodiment merely comprises anindependent technical solution. Those skilled in the art should have thespecification as a whole, and the technical solutions in each embodimentmay also be combined as appropriate to form other embodiments that canbe understood by those skilled in the art.

The present invention by no means is limited to the preferredembodiments described above. On the contrary, many modifications andvariations are possible within the scope of the appended claims.

What is claimed is:
 1. A method for measuring objects in a digestivetract based on an imaging system, comprising: simulating an environmentin the digestive tract and starting a calibration process of the imagingsystem; said calibration process, comprising setting a plurality ofcalibration points Q′ on a transparent enclosure of the imaging system;controlling a photographing unit of the imaging system to photograph andform a calibration image, and recording the calibration point Q′ imagedin the calibration image as an imaging point Q; calculating anddetermining a relationship between a relative angle θ of the calibrationpoint Q′ relative to the optical axis of the photographing unit and apixel distance Δq′ from the imaging point Q to the center of thecalibration image, and recording it as:θ=f(Δq′)  (1); calculating and determining a relationship between abrightness φ of any pixel in the calibration image and a depth distancez from an actual position of the pixel in the simulated digestive tractto a board where the photographing unit is disposed, and recording itas:z(x,y)=g(φ(x,y))  (2); calculating a relationship between a scale r ofany pixel in the calibration image and the depth distance z from theactual position of the pixel in the simulated digestive tract to theboard where the photographing unit is disposed, where scale is theactual length represented by unit pixel in the image, and recording itas:r=dz  (3); performing a measurement process after the calibrationprocess is completed; said measurement process, comprising placing theimaging system in the digestive tract; capturing and obtaining adetection image; determining a region in the detection image where thetransparent enclosure contacts with the digestive tract wall, andrecording it as a contact region, and setting at least one referencepoint P in the contact region, and recording the actual position of thereference point in the digestive tract as a target point P′; calculatinga pixel distance Δp from the reference point P to the center of thedetection image separately, and putting it into equation 1 to obtain arelative angle θ of the target point P′ relative to the optical axis ofthe photographing unit; calculating an actual distance from the targetpoint P′ to the board separately and recording it as depth distancez_(i); obtaining a predicted brightness g⁻¹(z_(i)) of the referencepoint P in the detection image according to equation 2 and the depthdistance z_(i); comparing the predicted brightness g⁻¹(z_(i)) of thereference point P with an actual pixel brightness img(P_(i)) of thereference point P to obtain a correction factor k_(i), and recording itas: $\begin{matrix}{{k_{i} = \frac{g^{- 1}\left( z_{i} \right)}{{img}\left( P_{i} \right)}};} & (4)\end{matrix}$ obtaining a mean value k of the correction factors k_(i)of all reference points P; calibrating all pixels in the detection imagewith the mean value k of the correction factors to obtain a depthdistance from an actual position of each pixel in the digestive tract tothe board, and recording it as depth image z(x, y), where:z(x,y)=g( k img(x,y))  (5); calculating a scale r of each pixel in thedetection image according to equation 3 and the depth image z(x, y);obtaining pixel coordinates S_(i) of each pixel point in the detectionimage, and calculating the actual two-dimensional coordinates S_(i)′ ofeach pixel in the detection image by the scale r; integrating to obtainactual three-dimensional coordinates (S_(i)′, z(x, y)) of each pixel;calculating or measuring a distance between any two pixels in thedetection image or the area within any range; wherein a photographingunit enclosed in the transparent enclosure, placed in between a boardand a front end of the transparent enclosure; the transparent enclosureis configured to be placed in the digestive tract and form the contactregion; and the image system comprises one or more computer processorsto calibrate and calculate.
 2. The method of claim 1, wherein the step“determining the region in the detection image where the transparentenclosure contacts with the digestive tract wall” comprises: selectingan edge part of the detection image away from the center of thedetection image; obtaining a brightness T of each pixel in the edgepart; gathering pixels in a region which the brightness T is greaterthan a threshold τ as the contact region.
 3. The method of claim 2,wherein the step “selecting the edge part of the detection image awayfrom the center of the detection image” comprises: marking an inner ringon the detection image that is centered on the center of the detectionimage, wherein the inner ring is close to the edge of the detectionimage and does not intersect; marking an outer ring on the detectionimage that is centered on the center of the detection image, wherein theouter ring intersects with the edge of the detection image; andrecording the part enclosed by the inner ring, the outer ring, and theimage edge as the edge part.
 4. The method of claim 1, wherein thedigestive tract comprises a plurality of regions and the imaging systemcomprises a plurality of exposure levels, and wherein, after the step“obtaining the mean value k of the correction factors k_(i) of allreference points P”, the correction factors are stored according todifferent exposure levels and different digestive tract regions.
 5. Themethod of claim 4, wherein after two or more mean values k of correctionfactors are obtained at the same exposure level and digestive tractregion, the average of the mean values k of correction factors iscalculated before storing and updating.
 6. The method of claim 1,wherein the step “calculating the actual distance from the target pointP′ to the board separately and recording it as depth distance z_(i) ofthe reference point P” comprises: obtaining the radius R of a frontenclosure of the transparent enclosure; calculating a distance R cos θfrom the target point P′ to the spherical center of the front enclosureseparately; obtaining the axial length H of an annular enclosure of thetransparent enclosure; calculating the depth distance z_(i)=R cos θ+Hfrom the target point P′ to the board separately.
 7. The method of claim1, wherein the step “obtaining the depth distance from the actualposition of each pixel in the digestive tract to the board” or“integrating to obtain the actual three-dimensional coordinates(S_(i)′,z(x,y)) of each pixel” further comprises: determining a value ofthe depth distance z of each pixel; when t₁≤z≤t₂, it is determined thatthe pixel is within an effective section of the detection image; whenz<t₁ or z>t₂, it is determined that the pixel is within an ineffectivesection of the detection image.
 8. The method of claim 7, wherein thestep “calculating or measuring the distance between any two pixels inthe detection image or the area within any range” is followed by:calculating a straight-line distance between any two pixels selected bya user in the effective section according to the three-dimensionalcoordinates of the two pixels; or, building a three-dimensional image ofany area according to the three-dimensional coordinates of pixels in thearea selected by a user in the effective section, and calculating astraight-line distance between any two pixels selected by the user fromthe three-dimensional image; or, calculating the area of any regionselected by a user in the effective section according to thethree-dimensional coordinates of the region; or, forming a scale in theeffective section, and marking graduations on the scale as those ofactual length; or, identifying the lesion region in the effectivesection automatically, and calculating the size or area of the region.9. The method of claim 7, wherein t₁=0, and t₂=60 mm.
 10. The method ofclaim 1, wherein the step “capturing and obtaining a detection image”comprises: controlling the imaging system to capture and obtain animage; correcting a radial distortion of the captured image and forminga detection image, and recording it as:img_out(x,y)=img_in(x(1+l ₁ R ² +l ₂ R ⁴),y(1+l ₁ R ² +l ₂ R ⁴))  (6);where, R=√{square root over (x²+y²)} represents the pixel distance fromthe pixel to the center of the detection image, l₁ and l₂ representdistortion parameters of the imaging system, x represents x-coordinateof the pixel, y represents y-coordinate of the pixel, img_in representsinput image, and img_out represent corrected image.
 11. A measuringsystem for objects in digestive tract based on an imaging system,comprising: one or more computer processors configured: to identify acontact region and a reference point P; to calculate the relationshipbetween a relative angle θ of a calibration point Q′ relative to anoptical axis of a photographing unit of the imaging system and a pixeldistance Δq′ from an imaging point Q to a center of a calibration imageand record as equation 1, and to calculate a relationship between abrightness φ of any pixel in the calibration image and the depthdistance z from the actual position of the pixel in a simulateddigestive tract to the imaging system, and record it as equation 2, andto calculate and determine the relationship between a scale r of anypixel in the calibration image and the depth distance z from the actualposition of the pixel in the simulated digestive tract to the imagingsystem, and record it as equation 3;θ=f(Δq′)  (1);z(x,y)=g(φ(x,y))  (2);r=dz  (3); to identify the brightness of any pixel or all pixels in thecalibration image or a detection image; to obtain equation 1 and thepixel distance Δp from the reference point P to the center of thedetection image to calculate the relative angle θ of a target point P′relative to the optical axis of the photographing unit, and to calculateactual distance from the target point P′ to a board where thephotographing unit is disposed and record it as depth distance z_(i),and to obtain the depth distance z_(i), equation 2 and the actual pixelbrightness of the reference point P to calculate a correction factork_(i), and to obtain equation 2 to calculate and obtain the depthdistance z(x, y) from the actual position of each pixel in the digestivetract to the board, and to obtain equation 3 to calculate the actualtwo-dimensional coordinates S_(i)′, and integrate to obtain the actualthree-dimensional coordinates of (S_(i)′, z(x, y)) of each pixel.